Semi-Supervised Regression using Spectral Techniques∗

Graph-based approaches for semi-supervised learning have received increasing amount of interest in recent years. Despite their good performance, many pure graph based algorithms do not have explicit functions and can not predict the label of unseen data. Graph regularization is a recently proposed framework which incorporates the intrinsic geometrical structure as a regularization term. It can be performed as semi-supervised learning when unlabeled samples are available. However, our theoretical analysis shows that such approach may not be optimal for multi-class problems. In this paper, we propose a novel method, called Spectral Regression (SR). By using spectral techniques, we first compute a set of responses for each sample which respects both the label information and geometrical structure. Once the responses are obtained, the ordinary ridge regression can be apply to find the regression functions. Our proposed algorithm is particularly designed for multi-class problem. Experimental results on two real world classification problems arising in visual and speech recognition demonstrate the effectiveness of our algorithm.